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Introduction 

In many admissions decisions a selection test is used to decide which 
members of an applicant pool will be selected for the available openings. 
The correlation coefficient between the selection test and the college's 
criterion of success, Grade Point Average, say, are often in the range of 
.AO to .55 for the applicants who are selected and then enroll. Such 
correlations are often squared and then interpreted as the. proportion of 
variance explained by the predictor. Such a statement is true but is very 
difficult for an admissions officer to relate to what the effect is on his 
entering class. Some better interpretive scheme is needed. 

The purpose of this paper is to show the effect of using a selection 
test on the average criterion score of the entering class. The correlation 
coefficient (times 100) is shown to be the percentage of improvement of 
using the selection test over what would happen on the average if the test 
were not used. Further, a simple formula is developed for approximating 
the probability that selection by chance would yield an entering class as 
^ able or abler than a class selected with the help of a valid predictor. 

^ It is further shown that, for reasonably sized colleges and with reasonable 

assumptions, selection using even small correlations will almost certainly 
^ result in an entering class that would earn hi^er criterion scores than 
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1. The Effect of Different Selection Proceduies 
Let us assume that an admissions officer has the task of filling n 
available openings in a college from a pool of N applicants. We will 
call the selection ratio f = n/N . We will assume throughout that his aim 
Is to select the "best" applicants in the sense that they are the students 
who will perform best on some criterion of college success such as grade 
point average (GPA) . The score on the criterion variable is not, of course, 
known at the time of admission decision and is never known for applicants 
who are not accepted and/or do not enroll. 

Let us now number the applicants with an index i (1-1,2,. ... ( N) and 
name the criterion variable y . The applicant pool is the population of 
interest;we will not assume that it is a sample from some larger popula- 
tion. The score of applicant i on the criterion variable y is y ± 
which is the score that the applicant would receive on the success criterion 
if the applicant were selected and did enroll. For convenience and without 
loss of generality, we will consider the values y ± to be in standard form, 
that is, to have an average value y of zero and a standard deviation S 
of one. We do not assume, at this point, that the distribution of the y ± 
is normal. 

Let us first consider what would happen in some extreme selection 
situations. First, let us assume that the admissions officer is prescient, 
that is, that he knows exactly how each applicant would perform on y if 
admitted and enrolled. In this case, the admissions process would be 
straightforward; the n candidates who would receive the highest y ± would 
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be accepted and in this way the average score on the criterion would be 
maximized. The decision rule is clear, except for tied y ± at the cut-off 
point which could be selected randomly. We will call the group selected 
this way the optimum group and the mean criterion score of that group will 
be called . Another extreme admissions situation is the case in which 

the admissions officer knows nothing about the applicant pool, at least 
nothing that is related to the success criterion. We can consider this 
case as knowing the applicants index number, i , and no more. In this 
case, the admissions officer might select applicants at random and hope 
for the best. There is a tiny probability that the resultant admittees 
would be the optimum group, but there is also a tiny probability of select- 
ing the group with the lowest possible average score on y . Yet, with 
no useful predictive information available, there would be little else 
that could be done. 

However, although we do not know what will happen if the applicants 
are selected at random, we do know everything that could happen. First, we 
know that there are precisely 



C = 



N! (1) 



n!(N - n)! 



different ways in which n students can be selected from a pool of N 
applicants. Let us number them with the index c (c=l,2, . . . ,C) . Each 
possible selection would^sult in a mean score for the criterion which we 
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will label y . We also know that the average of all possible y is 
c c 

ave(y ) = y » 0 < 2 ) 
c 

that is, not surprisingly, that the average of all samples of size n is 
the average of the applicant pool which is zero. We also know that the 
variance of the C possible y^ is 

since^the^ of the y 1 is unity. Proofs of the average and variance 

are shown in Cochran [1977]. If the student selection were at random, then 
any of the C groups is as likely to occur as any other, and thus the 
average mean criterion score over a large number of random assignments would 
approach zero. We will label the average mean as y Q . 

Let us now consider a less extreme — and more realistic — situation in 
which the admissions officer has some imperfect information about the appli- 
cants. Let us assume, for example, that each applicant's dossier contains 
his score on an admissions test. We will call that test x and the score of 
applicant i on that test x^ . is known for all applicants. For 

convenience, *e will also assume that the scores on x are in standard form 
with zero mean and unit variance. Let us also assume that, from experience, 
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we know that the linear correlation of x with y is p . We will assume 

throughout that p>0 . One possible admissions rule is to select the n 

students with the highest scores on x . The question we wish to explore 

here is how much better will the admittees be if selected using x rather 
than selecting at random. 

We do know something about what will happen if x is used for selection. 
First, we can partition the criterion scores into two parts 



y 4 = p Xi + e t W 



where the first part px ± is the "predicted" value of y ± and e ± is a 
residual. Let us call the average y ± for the selected group 7 p • * f we 
select the n applicants with the highest scores on x , then the value of 



- = - Z . x + i E e <*> 
y p n + i n + i 



where Z + means summing the scores of the applicants who have the top n 
values of x . If the relationship between y and x is linear, that is, 
the conditional mean of y given x is px , then the term ~^ e i will 
vanish and thus we may say 

where x is the average of the top n applicants. 
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Comparison of the average criterion score of selected groups under 
optimum information, , no information, y^ , and under information 

from some correlate, y , leads to an interesting interpretation of the 

P 

correlation coefficient which was first proposed by Brog^en [1946]. Under 
the assumption that the distribution of y is the same as the distribution 
of x , then the average of the top n scores on y is the same as the 
top n scores on x , although the actual persons having those scores may 
be different. Under this assumption, therefore, x + * y^ ? and ^ 

p = ^ (7) 

that is, the correlation coefficient (not squared) is the ratio of the mean 
of the group selected using x to the mean of the optimum group. Since the 
average mean of random selections is zero, lOOp may be interpreted as the 
percent of possible improvement in selection attained by using the selection 
test over what would happen, on the average, using random or any arbitrary 
selection procedure that is independent of y 

Figure 1 shows the relationships among y Q , y p , and y 1 ia the 
case where the top 10% of- the applicants are to be selected. The abscissa 
represents scores on y and the ordinate is proportional to the number of 
applicants with a particular score. The normal distribution was selected 
for pictorial purposes. The shaded portion of the curve represents the 
top 10% of the applicants which has a cut-off score at +1.28. The average 
score of the top 10% is a +1.755 which is the mean of the optimum group, 
y 1 . The average random selection, y Q , is zero. The line on the graph 
between ' y n and y- has tick marks to show where the mean of the selected 
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group would be if the correlation with the selection test, p , were 
+.10, .20, . . . , .90. If p « 0 , then the effect of the selection test would 
be the same as the y^ and if P = 1 , the effect is the same as 



Insert Figure 1 about here 
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2. The Probability of Exceeding y^ by Chance 
The improvement of the mean of the entering class using a small correla- 
tion, p = .10 or .20, say, may not, at first, seem worth the cost of the 
testing effort, but this is seldom so. The alternative would be to leave 
the selection to chance, since there is at least some probability that the 
chance selection would be better than the selection by a test. The question 

to be asked now is: what is the probability of actually selecting a sample 

by chance which results in an average y as high or higher than the mean 

of those selected by a test? The answer to this question is conceptually 

simple since all we would have to do is enumerate all C possible samples, 

count the number of samples which have means as high or higher than y^ , 

and divide that number by C to find the proportion as high or higher than 

7p . However, this direct solution is not feasible since even for a small 

problem like selecting 10 or 90 out of a sample of 100 has a value of C 

13 

is approximately 1.73 x 10 . However, a reasonable approximation to that 
proportion is possible and used here. 

The ability to compare random selection against selection by x gives 
an opportunity to show the importance of even small correlations in selecting 



FIGURE 1 
DISTRIBUTION OF APPLICANT POOL 



Y Q IS THE AVERAGE OF ALL RANDOM SELECTIONS 
Y 5 IS THE AVERAGE OF GROUP SELECTED WITH p = .5 
V, IS THE AVERAGE OF THE OPTIMUM GROUP 
/' f = n/N = .1 

CROSSHATCH AREA REPRESENTS TOP 10% OF DISTRIBUTION 
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an entering class. Figure 2 shows the probability of doing as well or 
better by chance than by a selection test using the assumption that the 
distribution of y in tjie unselected population was normally distributed 
with zero mean and unit variance. 

Probabilities are graphed for applicant pools of 100 and 1000, for 
different selection ratios, and for different values of p . We see that, 
in selecting 50 out of 100 applicants , -a correlation of .30 is sufficiently 
high that chance selection would yield a higher mean than selection using 
the test not more than one time in a hundred. If the correlation is .40, 
the selected group may range from 10 to 90 and the probability that a group 
selected by chance will not excel the predicted group will not exceed one 
in a hundred. With an applicant pool of 1000, a correlation of .10 is 
large enough so that the probability of selecting a group of 500 with a 
larger mean by chance selection is less than .01 and a correlation of .20 
is large enough so that the probability of selecting a group with a larger 
m ean by chance is about 1/1000 or less. Thus, the use of a test with even 
a modest correlatibn will almost certainly result in a higher average of y 



Insert Figure 2 about here 



Figure 3 shows why even such small correlations are useful in selec- 
tion. Superimposed on Figure 1 is the approximate distribution of y c 
for the case where the applicant pool U 100 and but 10 are to be selected. . 
The mean of this distribution is zero and the! standard deviation is 
/09 » .3 . The distribution of 7 C ^ very tall near its mean when compared 
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FIGURE 2 

PROBABILITY OF 7 e > y, FOR VARYING N,n and p 



APPLICANT POPULATION « 100 




NUMBER SELECTED 
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APPLICANT POPULATION = 1000 
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to the distribution cf y^ which indicates that most of the y c are 
close to the average and there are few large deviant values. It is true 
that the maximum value of y is , but the probability of this 

occurring by chance is approximately .577 x 10 It is to this distri< 

bution of V that the value y should be compared and it is clear 

J c P 

that even for this very small problem the admissions officer is unlikely 

to improve on y by chance. For more realistic situations where the 

P _ 
applicant pool is much larger, the distribution of y c is even more 

spiked and thus the probability of exceeding y p by chance is even less 

likely for many reasonable values of p 

Insert Figure 3 about here 
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FIGURE 3 

DISTRIBUTION OF MEANS OF RANDOM SELECTIONS 

N=100 n=10 
DOTTED LINE REPRESENTS DISTRIBUTION OF ALL y 




♦ # • - ----- — -— "- - ■ 

13 



References 

Brogden, Hubert E. On the interpretation of the correlation coefficient 
as a measure of predictive efficiency, ^rjiaj^^uc^onal 
Psychology, 1946, 37, 65-76. 

Cochran, William G. Ss^AlngJTec^ques (third edition), New York: 
John Wiley & Sons, 1977. 



It 



ERIC 



